[5/10] libs/libc/elf: Load an FDPIC object. - #19942
Conversation
❌ Cross-repo dependency could not be appliedThe Build report says the declared dependency PR(s) could not be applied, so CI did not run against the combined code:
Reason: cherry-pick failed (if your PR has merge commits, rebase instead) CI run: https://github.com/apache/nuttx/actions/runs/32671745418 |
|
4b54f93 to
ae1aa73
Compare
❌ Cross-repo dependency could not be appliedThe Build report says the declared dependency PR(s) could not be applied, so CI did not run against the combined code:
Reason: cherry-pick failed (if your PR has merge commits, rebase instead) CI run: https://github.com/apache/nuttx/actions/runs/32832058242 |
ae1aa73 to
aed7275
Compare
1909f02 to
1667e86
Compare
🔗 Cross-repo PR dependenciesThe read-only Build run reported the following dependent PR(s) and fetched head SHA(s): CI run: https://github.com/apache/nuttx/actions/runs/32989079270 |
1667e86 to
5df7e8f
Compare
🔗 Cross-repo PR dependenciesThe read-only Build run reported the following dependent PR(s) and fetched head SHA(s): CI run: https://github.com/apache/nuttx/actions/runs/33072459903 |
|
@casaroli please rebase your patch to fix the conflict. |
The commits that follow teach the ELF loader to load an FDPIC object. This puts the option they hang off and the definitions they share in one place first, so each of them builds on its own. CONFIG_FDPIC depends on ARCH_HAVE_ELF_FDPIC, which an architecture selects when it has a PIC base register and the FDPIC relocations. Only armv7-m and armv8-m select it today, and it defaults off, so nothing changes for anyone who does not ask for it. include/nuttx/fdpic.h holds what both sides of the loader need: the two word function descriptor an FDPIC module passes instead of a code address, the test for whether the caller is such a module, and the call sequence that enters one with its own data base. All of it is behind CONFIG_FDPIC, thus the header is empty without it and a file may include it unconditionally. The call sequence itself is architecture specific, so arch/arm/include/arch.h supplies it as up_fdpic_invoke(), beside the other PIC base register macros. up_setpicbase() cannot serve here: the register has to hold the module's base for exactly one call and then go back, and nothing in C tells the compiler the register is live across that call, so the save, the install, the branch and the restore have to be one sequence. Built for mps3-an547:bl and mps3-an547:picostest, with CONFIG_FDPIC off, which is every configuration in the tree. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
An ET_DYN object is loaded into one allocation with its data behind its text, because its data references sit at a fixed distance from the code that makes them. An FDPIC object does not work that way: it reaches its data through a base register, so the two segments can be placed wherever suits, and the point of the format is that the read-only one is left on the media and executed there while only the writable one is copied. One copy of the text then serves every instance. So libelf_load() grows a second case. The object announces itself in the OS/ABI byte, which is noted once in libelf_loadhdrs() rather than re-derived; e_flags cannot be used for this, as an FDPIC object's are an unremarkable EABI version and testing them would reject every valid module. Text is taken from the media address plus the segment's own file offset -- the same arithmetic the ET_REL path already does with sh_offset -- and libelf_loadfile() does not read it. If the filesystem cannot show its media, the loader copies the text to RAM instead. The module then loses the shared text and the flash saving, but it runs. Obtaining that address needs two mechanisms, and they are not interchangeable. A compacting filesystem can move a file's blocks, so it hands out an address only with a pin that holds them still and expects the pin back; xipfs is the one in tree. A filesystem whose layout never changes has nothing to hold and answers FIOC_XIPBASE with a bare address; romfs and tmpfs are those. libelf_xipacquire() asks for the pin first, because a filesystem that needs one is not safe without it, and libelf_unload() gives it back. The loader asks for a pin only if it can hold one, or the pin would stay for ever. The pin is thus not specific to FDPIC. Any module that executes in place from a compacting filesystem takes one, and gives it back at unload. mmap() is not used, though both filesystems implement it. The mapping would be recorded against whichever task called the loader, while the release happens when the module's own task exits, which is a different group -- so the pin would outlive the module and the extent would never become movable again. Unloading has to change with placement: the existing path frees only textalloc because ET_DYN had a single allocation, which would leak an FDPIC object's data and free media the filesystem only lent us. Nothing here runs for a non-FDPIC object; every branch is behind the flag and the single-allocation path is untouched. Built and booted mps3-an547:picostest, which is CONFIG_ELF with CONFIG_PIC, with no change in behaviour. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
libelf_relocatedyn() reads the handful of DT_* tags it needs to walk the relocation tables and ignores the rest. Three more matter now. DT_PLTGOT is where the object's data base lives. An FDPIC module runs with that in the PIC base register, and every function descriptor built for it names the same base as the one its callee should run with, so without it there is nothing to put in a descriptor's second word. The DT_*_ARRAY tags are the constructor and destructor tables. These are already found through the section headers a few lines further down, and that path is kept, but the dynamic tags are the authoritative copy and an object is not obliged to carry section headers at all. Both paths now translate through libelf_addr(), so they agree on the answer rather than depending on which ran last. The tag values themselves were missing from include/elf.h and are added. Sizing the descriptor pool has to happen here rather than later. R_ARM_FUNCDESC asks the loader to manufacture a descriptor and hand back its address, which means the space must exist by the time the relocation is applied, and by then the segment has been placed. So libelf_elfsize() reserves it behind the writable data, bounded by the relocation count -- one relocation cannot ask for more than one descriptor. That bound has slack in it, but a descriptor is two words and modules are small, which is cheaper than walking every relocation twice to get an exact count. Nothing here runs for a non-FDPIC object. Built and booted mps3-an547:picostest with no change in behaviour. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Running one for the first time turned up two holes in the ET_DYN path. Neither shows up in a build. An undefined symbol is resolved with libelf_findglobal(), which searches only the table of globally registered symbols. The export table that exec() hands its caller went no further than the ET_REL path, so an ET_DYN module could not import anything the caller supplied. Invisible while such modules resolved everything internally; an FDPIC module imports its libc, and every import failed with "Unable to resolve addr of ext ref printf" although the caller had passed a table containing printf. The export table is now threaded into libelf_relocatedyn() and consulted when the global table has no answer, leaving the existing lookup order intact. A relocation naming a symbol defined inside the object was dropped silently. The code handles a relocation with no symbol, and one against an undefined symbol, but a defined symbol fell through both. That was harmless while every dynamic relocation arriving here had symbol index zero, which is the case for R_ARM_RELATIVE. FDPIC brings the first ones that do not: a pointer to a static function is emitted against the *section* symbol, so the value is the section base and the offset within it -- including the Thumb bit -- is carried as the addend. Deriving a value from the word being patched, as the no-symbol case does, would translate that addend as though it were an address. Confirmed against a real module: .text at 0x23c plus an addend of 0x95 gives 0x2d1, which is the function with its Thumb bit. Also stop libelf_symname() reporting a nameless symbol as an error. A section symbol has no name, and libelf_findsymbol() walks the whole table looking for optional entries such as nx_stacksize, so it meets these routinely and checks for -ESRCH itself. At error level it printed ten or more lines per module load and buried the diagnostics that matter. Built and run on lm3s6965-ek with the examples/elf ROMFS. The ET_REL test modules load as before, and an FDPIC module now loads, relocates, resolves printf and puts from the table exec() supplied, and calls through a function descriptor of its own. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
A module that dlopen()s a library gets back function addresses from dlsym() and calls them. Under FDPIC a bare code address is not enough: the callee needs its own data base as well, so what dlsym() returns has to be a function descriptor. The exported symbol table carries no type information -- symtab_s is a name and a value, and its own comment says typing would have to be added to support anything but function pointers -- so by the time dlsym() is asked there is no way to tell a function from an object. libelf_insertsymtab() is the last point that can: st_info is still in hand there. So an FDPIC object's exported functions are published as the address of a descriptor carved from the module's pool, and dlopen(), dlsym() and the module registry need no knowledge of FDPIC at all. The pool is sized for the dynamic symbol table as well as the relocations, since both can draw from it. That leaves the symbol values themselves, which were wrong for any ET_DYN object. libelf_loadsymtab() adds the symbol's section address to its value, which is right for ET_REL, where the section address is where the section was actually placed and the value is relative to it. In a shared object both are already full link-time addresses, so adding them counts the section twice. It needs translating onto wherever the object was placed instead. Library data is shared between everything that dlopen()s it, because the registry holds one instance per name. Giving each user its own copy would mean teaching the registry about instances, which is a much larger change to shared code; an executable loaded through exec() already gets its own data, since that path loads a fresh copy each time. Built and run on lm3s6965-ek with the examples/elf ROMFS; the FDPIC module continues to load, relocate and call through its own descriptors. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
The FDPIC work touches these files, and nxstyle reports errors on the lines around every hunk, which fails the check job. The errors are older than this series: a switch body indented two columns too deep in elf_symbols.c, and declarations with no blank line after them. Whitespace and one reworded comment, no change in behaviour. Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
5df7e8f to
cc2b937
Compare
Summary
Fifth of the PRs that #19673 is being split into. This is the loader side: enough for the ELF loader to place an FDPIC object and bind it. The ARM relocations are
[6/10], the callback entry points[7/10], theexec()path[8/10],DT_NEEDED[9/10], documentation and a board configuration[10/10].An FDPIC object places its two
PT_LOADsegments independently, so its read-only segment runs where the filesystem already holds it and only the writable segment is copied to RAM, once per running instance. Several instances of one module therefore share one copy of the text. A filesystem that cannot show its media gets the text copied to RAM instead, so the module still runs but shares nothing.The first commit adds
CONFIG_FDPIC,ARCH_HAVE_ELF_FDPICandinclude/nuttx/fdpic.h, so the four that follow each build on their own.The read-only segment needs the filesystem to hold its blocks still. A compacting filesystem such as XIPFS gives its media address together with a pin, which the loader holds through a file reference, because the unload runs on a different task from the load.
The last commit is style only. CI feeds nxstyle the diff hunks with three lines of context, so style errors that are older than this series, in the lines around every hunk, fail the check job. They are a switch body indented two columns too deep, an initializer brace one level in, and declarations with no blank line after them.
Impact
CONFIG_FDPICdefaults off and no in-tree configuration sets it, so nothing changes for anyone yet. With it off the four loader commits compile to what they were.[4/10]#19941 has merged, so this is six commits on master.Testing
mps3-an547:picostestbuilds withCONFIG_FDPICboth on and off, andmps3-an547:blbuilds, which is every configuration in the tree since none sets the option.tools/checkpatch.sh -c -u -m -gpasses over the whole branch.mps3-an547:blis the one to keep an eye on: it setsCONFIG_ARCH_USE_SEPARATED_SECTION, andCONFIG_LIBC_ELFpulls inARCH_USE_TEXT_HEAP, which selects the three argumentup_textheap_memalign(). #19673 calls the two argument form there and fails to build; that is fixed in the commit that adds the call.There is no hardware for this series, so the runtime evidence is emulated. A module cannot be relocated until
[6/10], so the runs that matter are the umbrella's, on QEMUmps2-an500: 131/131, 34/34 and 7/7, with the logs in #19673.